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This study explores the correlation between technology utilization and 
language acquisition while analyzing the impact of moderating variables on 
this relation. Our meta-analysis approach analyzes data from 43 extracts out 
of 19 primary studies published between 2012 and 2021. Our data analysis 
employs a random-effect model utilizing a significance level of a = 0.05. 
Additionally, the authors examine four moderating variables: level of 
education, location of research, proficiency in language, and year of 
publication. Technology-based language acquisition outperforms traditional 
methods, indicating a significant and moderate impact on the learning process. 
This study enhances comprehension of the efficacy of technology in language 
acquisition by identifying various factors, such as the geographical location 
of research, methods of assessing language proficiency, and technology type 
employed. However, there is insufficient evidence to support the notion that 
educational level or sample size significantly impact technology-based 
language acquisition. This meta-analysis highlights the importance of 
considering nuanced factors when integrating technology into language 
learning. The findings emphasize the possibility of technology to transform 
methods of acquiring language and urge additional investigation into 
customized strategies that optimize its advantages. 


This is an open access article under the CC BY-SA license. 


Corresponding Author: 


Syarief Fajaruddin 


Department of Educational Research and Evaluation, Universitas Negeri Yogyakarta 


Colombo Street No. 1, Yogyakarta 55281, Indonesia 
Email: syarieff@uny.ac.id 


1. INTRODUCTION 

Global development has been significantly influenced by technology, particularly in education. The 
advancement of technology in the field of education has manifested in the implementation of more 
sophisticated learning facilities. The accessibility of technology has aided educators in enhancing their 
instructional quality. The statement has been delivered by Keengwe and Georgina [1] as they argue that 
technological advancement has brought about significant changes in teaching and learning practices. 
Information technology can be accepted as a medium for conducting the educational process, including aiding 
the teaching-learning process, which includes reference searching and information sourcing [2]. 

Recent science and technological development has significantly influenced the domains such as 
economy, society, politics, law, art, culture, and even education [3]. In education, especially in language 
learning, the technology significantly influences the success of the language learning process 
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and results [4]-[6]. The findings from a Hussain [6] study show that Information and Communication 
Technologies (ICT)-based learning can improve vocabulary mastery. Similarly, according to Enayati and 
Gilakjani [5], using computer-assisted language learning (CALL) can improve vocabulary mastery compared 
to the traditional method. Departing from the results of these studies, it is clear that technology can improve 
language learning. 

Several meta-analysis studies have been conducted concerning the association between technology 
and language learning. Zhao [7] has studied the most recent technological and language-learning 
advancements. In his research, he utilized journal articles published between 1997 and 2001. According to the 
results of his study, the literature on the effectiveness of technology use in language education is limited in 
four ways: the small number of systematic empirical evaluation studies, the limitation of the study to higher 
education and adult learners, the limitation to the general foreign language and English, and the short-term 
experiment with one or two learning aspects (such as grammar or vocabulary). 

Grgurović et al. [8] conducted a study on the effectiveness of computer-supported language learning. 
The researchers utilize 37 studies published between 1970 and 2006 in academic journals in this study. These 
37 articles come from three electronic databases: linguistics and language behavior abstracts (LLBA), 
education resources information center (IRIC), and dissertation abstracts (DA), as well as six periodicals: 
CALL, System, computer assisted language instruction consortium (CALICO) Journal, ReCALL, language 
learning and technology (LL&T), and TESOL quarterly. 

On the contrary, [9]-[12] have conducted a meta-study by analyzing the influence of cellular gadgets 
on language learning. The study by [11] uses nine journal articles, conference proceedings, and doctoral 
dissertations published between 2008 and 2018. The results of their study show that the use of the cellular 
phone for language learning is more effective than the conventional method. Then, another study by Cho et al. 
[12] used 20 studies from electronic databases such as education resources information center (ERIC), 
EBSCOhost (academic research complete), PsycINFO, journal storage (JSTOR), and ProQuest Dissertations 
and Theses from 2005 until 2017. The results of their study show that the overall size effect is 0.51. Next, Taj 
et al. [10] study utilizes thirteen studies published between 2008 and 2015 from the electronic databases ERIC, 
digital libraries of the University of Jeddah, digital libraries from the University of Malaysia Pahang, and 
Google Scholar [10]. The findings of their investigation indicate that mobile assisted language learning 
(MALL) has promoted English as a foreign language (EFL) instruction. Their study's aggregate size effects 
(d = 0.8) are the largest of all. In the meantime, Sung et al. [9] utilized 44 journal articles and doctoral 
dissertations published between 1993 and 2003. Their study reveals a moderate effect size of 0.55 regarding 
the use of mobile devices in language acquisition. These meta-analysis investigations were conducted using 
out-of-date sources from 1970 to 2017. In order to conduct further research on the relationship between the use 
of technology for language learning and the effect of potential moderating variables that influence the strength 
of the relationship, it is necessary to analyze the overall impact of the most recent technology on language 
learning. 


2. METHOD 
2.1. Literature search 

The research was conducted by searching the Scopus database for articles addressing the topic of 
technology-based language learning published in scientific or academic journals as of March 3, 2021. Utilizing 
the keywords "technology-based and language learning" and "the effect of technology on language learning," 
a literature review was conducted. To ensure the robustness and reliability of our meta-analysis, strict inclusion 
and exclusion criteria were applied to the selection of studies. These criteria were established a priori to 
maintain consistency and minimize bias in the selection process. Table 1 provides a detailed overview of the 
inclusion and exclusion criteria. 

Been studying has collected 291 documents. 29 out of 291 documents have been collected since 
researchers imposed the following restrictions on these documents: i) the documents are from open access; ii) 
the documents are from the period of 2012—2021 (specifically on May 2021 since 201 has not ended yet so that 
it becomes possible the number of indexed articles may expand after May; iii) the documents are of the social 
subject; iv) the documents are of article type; v) the documents are in the final publication stage; vi) the 
documents are from the journal sources; and vii) the documents are in English. Afterwards, the researchers 
search the articles by using the keyword "the effect of technology on language learning." The results of the 
search were returned in 1,673 documents. With similar limitations, the researchers have gathered 85 
documents. 

Several screening criteria will be applied to the documents obtained through the literature search. First, 
the studies should assess the correlation between technology use and language learning and report the 
measurement outcomes. In addition, the investigations should report the relationship between the variables. 
Consequently, studies that only report the outcomes of technology use or language acquisition will be excluded. 
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Similarly, studies that do not report the relationship between the variables will also be excluded. Second, the 
studies should explicitly report the sample size used. 


Table 1. Inclusion/exclusion criteria for studies in the meta-analysis 


Criteria Included Excluded 
Open access All open access Gold, hybrid gold, bronze, green 
Years 2011-2021 Other 
Subject area Social science Other 
Document type Article Conference paper, conference review, review, book chapter 
Publication stage Final Article in press 
Source type Journal Conference proceeding, book series, book 
Language English Other 
Research design Quantifiable comparisons (posttest control Other 

and experimental groups) 

Data provided N, M, SD, or a way of extracting equivalent Others 
Quality All Ø 


Consequently, the studies that do not report the sample size will be excluded. Third, the studies should 
report the technique for measuring the technology use in language learning. Hence, the studies that only report 
the association between technology use and language learning with a clear description of how the technology 
used in language learning has been measured will be excluded. Fourth, each study should report Pearson’s r 
correlation. Thus, the studies that only report the regression model analysis reports, or the multilevel regression 
model analysis reports will be excluded. Fifth, the studies should use English. Sixth, the grade level of 
participants consists of elementary school until college level. Therefore, the studies in which participants are 
from the preschool level will be excluded. 

The number of documents that meets the inclusion criteria is 19 main studies as shown in Figure 1. 
From the 19 main studies, the researchers have found 43 independent samples that will be analyzed. This figure 
appears to the surface because the 43 independent samples from the 19 main studies report proficiency of more 
than one language aspect [13]—[20]. Meanwhile, other studies have only reported on one aspect of language 
proficiency [5], [6], [21]-[29]. 


Standardised Mean 
Study Difference Effect Size 95%-Cl Weight 
Hosseinpour et al. [21] — 0.23 [-0.31;0.77] 23% 
Abdulrahman et al. [22] —— 0.10 [-0.41;0.60] 24% 
Lopez [23] ——— 1.08 [0.35;1.82] 1.8% 
Mellati and Khademi [29] ——— 1.29 [0.77;1.80] 2.4% 
Hussain [6] im 0.61 [0.05; 1.17] 23% 
Enayati and Gilakjani [5] —E—— 0.66 [0.15;1.17] 24% 
Faroog et al. [24] on 0.30 [-0.30; 0.90] 22% 
Schechter et al. (1) [13] -a 0.26 [-0.17;0.69] 2.6% 
Schechter et al. (2) [13] — 0.04 [-0.39;0.47] 2.6% 
Schechter et al. (3) [13] —s 0.34 [-0.09; 0.78] 2.6% 
Parvez et al. (1A) [14] = 0.73 [0.44; 1.03] 3.0% 
Parvez et al. (1B) [14] + 0.80 [0.50; 1.09] 3.0% 
Parvez et al. (1C) [14] + 0.53 [0.24;0.81] 3.0% 
Parvez et al. (2A) [14] = 1.10 [0.80;1.41] 3.0% 
Parvez et al. (2B) [14] HE 0.80 [0.50; 1.09] 3.0% 
Parvez et al. (2C) [14] — 0.41 [0.13;0.70] 3.0% 
Elverici (1) [15] ——_+—_ 290 [1.91;3.89] 13% 
Elverici (2) [15] E -0.25 [-0.94;0.43] 1.9% 
Kurt (1) [16] + 0.68 [0.07; 1.28] 21% 
Kurt (2) [16] > 0.51 [-0.09;1.11] 2.1% 
Awada [25] =g 2.06 [1.39;2.73] 2.0% 
Kashani et al. (1) [17] —— 0.69 [0.19;1.19] 2.4% 
Kashani et al. (2) [17] +S 0.39 [-0.11;0.88] 24% 
Kashani et al. (3) [17] an cae 0.15 [-0.34;0.64] 25% 
Kashani et al. (4) [17] —_#— 0.02 [-0.47;0.51] 25% 
Kashani et al. (5) [17] —_s—_ -0.05 [-0.54;0.44] 25% 
Kashani et al. (6) [17] +a 0.33 [-0.16;0.82] 2.5% 
Liu et al. [26] — 0.53 [0.10;0.95] 2.6% 
Yesilbab and Korkmaz (1) [18] a -0.09 [-0.60;0.42] 24% 
Yesilbab and Korkmaz (2) [18] + 0.16 [-0.34; 0.67] 24% 
Yesilbab and Korkmaz (3) [18] + 0.08 [-0.43; 0.59] 24% 
Yesilbab and Korkmaz (4) [18] + 0.63 [0.11;1.15] 2.4% 
Yesilbab and Korkmaz (5) [18] ae a 0.25 [-0.26;0.76] 24% 
Abu-Hardan et al. (1) [19] + —_— 1.43 [0.61; 2.25] 1.6% 
Abu-Hardan et al. (2) [19] —_ 1.20 [0.34; 2.06] 1.5% 
Abu-Hardan et al. (3) [19] ++ 0.78 [-0.21;1.77] 1.3% 
Abu-Hardan et al. (4) [19] a EE oo 0.66 [-0.46; 1.79] 1.1% 
Abu-Hardan et al. (5) [19] O 1.28 [0.80;1.76] 25% 
BavaHarji et al. (1) [20] E 0.51 [0.10;0.93] 27% 
BavaHarji et al. (2) [20] —= 0.62 [0.20;1.03] 27% 
BavaHarji et al. (3) [20] —_ 0.59 [0.18; 1.01] 27% 
Alemi et al. [27] + 0.72 [0.10;1.33] 2.1% 
Alfaleh [28] —— -0.45 [-1.39;0.49] 1.4% 
Random effects model i 4 => : : : : 0.56 [0.42; 0.71] 100.0% 
A 0 1 2 3 4 


Figure 1. Forest plot for the 43 independent samples 
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2.2. Included samples 

The 19 primary studies for our meta-analysis originated from a reputable database, as detailed in 
Table 2. These studies were selected to provide a comprehensive overview of the diversity of research 
conducted in the field of technology-based language acquisition. The inclusion criteria ensured a wide range 
of research contexts and methodologies, which contributed to the robustness of our analysis. To ensure the 
relevance and novelty of our findings within the pertinent time frame, our literature analysis covered studies 
published from 2012 to 2021. This allowed us to capture recent developments in the field of technology- 
assisted language learning. 


Table 2. Database of the main study 


No. Study Journal Publisher 
1. Hosseinpour et al. [21] Turkish Online Journal of Distance Education Anadolu University 
Z; Abdulrahman et al. [22] International Journal of Language Education Universitas Negeri Makassar 
3: López [23] Estudios de Linguistica Inglesa Aplicada Universidad de Sevilla, Servicio de 


(ELIA) 


Publicaciones 


4. Mellati and Khademi [29] International Journal of Applied Linguistics Australian International Academic Centre 
and English Literature 

5. Hussain [6] International Journal of Education Universitas Pendidikan Indonesia 

6. Enayati and Gilakjani [5] International Journal of Language Education Universitas Negeri Makassar 

7. Farooq et al. [24] English Language Teaching Canadian Center of Science and Education 

8. Schechter et al. [13] Computers in the Schools Routledge: Taylor and Francis Group 

9; Parvez et al. [14] Sustainability MDPI (Multidisciplinary Digital Publishing 
Institute) 

10. Elverici [15] Turkish Online Journal of Distance Education Anadolu University 

11. Kurt [16] Elementary Education Online Ankara University Faculty of Education 
Department Primary Education 

12. Awada [25] Cogent Education Taylor and Francis 

13. Kashani et al. [17] English Language Teaching Canadian Center of Science and Education 

14. Liuetal. [26] Symmetry MDPI (Multidisciplinary Digital Publishing 
Institute) 

15. Yeşilbağ and Korkmaz Education and Information Technologies Springer 

[18] 
16. = Abu-Hardan et al. [21] International Journal of Learning, Teaching Society for Research and Knowledge 
and Educational Research Management 

17. BavaHarji et al. [20] English Language Teaching Canadian Center of Science and Education 

18. Alemi [27] International Education Studies Canadian Center of Science and Education 

19. Alfaleh [28] The 2015 WEI International Academic Elsevier 


Conference Proceedings 


2.3. Data analysis 
— Effect size calculations 

For the correlation study, the Pearson coefficient of correlation (r) refers to the index of effect size 
from a study [30]-[33]. If the individual studies are statistically significant, then the resulting effect size is 
assumed to use using a p-value of 0.05. In order to ensure the stabilization of the sample distribution, the r 
should be transformed into the Fisher’s z transformation [34], [35]. The formula that will be used for 
transforming r to z can be viewed in (1): 


r+1 


z=0.5x (=) (1) 

After the mean effect size and the trust interval have been attained, these values are transformed into 
r. In order to calculate the mean effect, the researchers use the random effect model. The random effect model 
is selected because the researchers assume that the effect size of the different studies probably comes from the 
diverse populations, and the diverse populations have their respective sample distribution [35], [36]. The 
diversity of the experimental settings from each study (grade level, country or region, and gender) will be fitter 
in the analysis if the random effect model is used [37]. In addition, the meta-study analysis data analysis will 
use Jeffrey’s amazing statistics program (JASP) free software. 


2.4. Moderator analyses 

In order to identify the variation among the results of the different studies, the researchers also conduct 
the heterogeneity test (Q test). If the Q statistics are significant, it can be concluded that each study does not 
come from the common population. In other words, the significant Q statistics show that the mean effect size 
of each component in the moderating variable is significantly different, and, therefore, the moderating variable 
analysis can be potentially conducted. 
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In the study, the five moderating variables (grade level, region, measure of self-control, measure of 
academic achievement, and sample size) are analyzed using the analysis of variance (ANOVA)-like models. 
For the ANOVA-like models, the researchers report the within-group effect means (weighted r), the 95% 
confidence intervals (CI), the within-group variability (Qw), and the between-group heterogeneity (Qb). The 
significant Qb statistics show that the mean effect size between the components in the moderating variables is 
significantly different. 

Next, the moderating variable publication year is analyzed by using Pearson's correlation test. The 
analysis is conducted in order to identify the association between the publication year and the effect size. The 
researchers report the coefficient of correlation r and the 95% confidence intervals within the analysis. The 
significant r correlation shows a significant relationship between the publication year and the effect size. All 
the moderating variables are analyzed using assistance from the JASP free software. 


2.5. Evaluation of publication bias 

The meta-analysis study uses three approaches in exploring the publication bias: funnel plot, Egger’s 
test, and fail-safe N. The funnel plot is used to display all effect sizes clearly. If the pattern that has been shaped 
is symmetrical, then the pattern will indicate that there is not any biased publication [36]. Then, Egger’s test is 
a linear regression method used to test the symmetrically of the funnel plot [38]. The fail-safe N estimates the 
number of studies with statistically insignificant results (unpublished data) required for the mean effect size to 
become statistically insignificant [39]. 


3. RESULTS AND DISCUSSION 
3.1. Results 

The researchers collected data from 19 main studies (scientific articles) that met the predefined 
inclusion criteria. In total, 43 independent samples were obtained, and these samples became the objects of our 
meta-analysis. Table 3 (see in Appendix) presents key information about these studies, including year of 
publication, effect size (g), standard error (SE), publication type, sample size, and grade level. Table 4 (see in 
Appendix) provides information on the geography and technology categories utilized in language-learning 
studies. It is noteworthy that a majority of these studies were conducted in Turkey (20.93%), and online media 
(37.21%) was found to be the most extensively used technology category for language-learning activities. 
Additionally, Table 5 (see in Appendix) illustrates the moderating variables analyzed in our study. These 
variables encompass a range of factors, including level of education, location of research, measurement of 
language proficiency, and type of technology used. 


3.1.1. Findings from the main analysis 

The main findings from the meta-analysis study are displayed in Table 6. The analysis results that 
have been conducted using the random effect model show that the mean effect size from the 43 studies is 0.562 
(p < 0.001), with the interval degree of trust 95% from 0.418 until 0.706. These results imply that there has 
been a significant impact from the use of technology on language learning activities in comparison to traditional 
learning (the learning process without the use of technology). The size effect values of 0.80, 0.50, and 0.20 
represent the big, moderate, and small sizes [40]. Thereby, it can be concluded that the effect of technology on 
language learning belongs to the “Moderate” category compared to traditional learning. 

The results of the heterogeneity test as show in Table 6, that the effect size of the 43 independent 
samples has been heterogenous (Q = 138.055, df= 42, p < 0.001). These results show that the inter-effect size 
variance used in the study has been very varied. Thus, these results imply the need to analyze the moderating 
variable to identify each moderating variable's contribution toward the inter-effect size variance differences. 

Figure | displays the forest plot of 43 studies analyzed using the random effect model. In the forest 
plot, the effect size of each study is symbolized by square dots, while the horizontal line on both sides of the 
squared dot displays the estimates of trust interval. The forest plot shows that the effect size of the 43 studies 
has been quite varied, with the smallest effect size of -0.45 and the largest effect size is 2.90. Several studies 
have negative effect sizes, such as [15], the second study by Kashani et al. [17], the fifth study by Yeşilbağ 
and Korkmaz [18], and the first study by Alfaleh [28]. These studies indicate that the implementation of 
technology in language learning activities within these studies has been proven ineffective compared to 
traditional language learning activities. In addition, several studies have also shown insignificant size effects 
[21], [22], [24]. These findings indicate that several studies confirm that the impact of technology use on 
language learning activities is not significantly different from that of traditional language learning. However, 
in general, it can be seen in the forest plot that most of the studies that have been analyzed have high and 
significant side effects [14], [23], [29]. These studies indicate that using technology in language learning is 
more effective than traditional learning. 
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Table 6. The impact of the use of technology in language learning: overall results and moderating variable 


analysis 
Variable K Mean SE 95% CI Q df Qw Qb 
Overall 43 0.562* 0.074 [0.418, 0.706] 138.055* 42 
Degree 3 133.007 5.048 
Elementary school 16 0.484* 0.085 (0.317, 0.651] 41.89 15 
Junior high school 2 0.558** 0.173 [0.219, 0.896] 0.056 1 
Senior high school 8 0.979** 0.335 [0.323, 1.636] 40.631 7 
University 17 0.536* 0.122 [0.297, 0.775] 50.43 16 
Sample size 1 136.97 1.085 
Big 21 0.553* 0.078 [0.401, 0.705] 59.11 20 
Small 22 0.605* 0.145 [0.320, 0.889] 77.86 21 
Country 7 85.622 52.433 * 
Indonesia 2 0.337 0.258 [-0.168, 0.843] 1.801 1 
Iran 7 0.649* 0.109 [0.436, 0.862] 8.817 6 
Jordania 5 1.180* 0.172 [0.846, 1.513] 1.961 4 
Malaysia 6 0.251** 0.109 [0.036, 0.465] 5.78 5 
Pakistan 7 0.696* 0.096 [0.508, 0.883] 14.65 6 
Turkey 9 0.480 0.263 -0.035, 0.996 35.463 8 
United States of America 4 0.171 0.123 -0.070, 0.412 2.778 3 
Others 3 1.196** 0.456 [0.303, 2.089] 14.372 2 
Skills under measurement 6 119.747 18.308** 
General 5 0.391* 0.113 [0.170, 0.611] 1.818 4 
Listening 3 0.196 0.216 -0.226, 0.619 3.744 2 
Reading 5 0.436 0.251 -0.056, 0.928 16.142 4 
Writing 10 0.498** 0.199 [0.108, 0.888] 44.505 9 
Vocabulary 11 0.659* 0.115 [0.434, 0.885] 14.36 10 
Sign language 6 0.725* 0.098 [0.533, 0.917] 12.83 5 
Others 3 1.057 0.919 -0.745, 2.859 26.348 2 
Technology in use 4 107.858 30.197* 
Computer 9 0.496* 0.132 [0.237, 0.755] 19.1 8 
Mobile application 7 0.724* 0.091 [0.547, 0.902] 12.83 6 
Online media 16 0.523* 0.109 [0.310, 0.737] 34.18 15 
Social media 3 1.546 0.942 [-0.300, 3.392] 34.474 2 
Others 8 0.261** 0.095 [0.074, 0.448] 7.274 7 


Note: *p < 0.001; **p < 0.050; k = the number of studies; CI = confidence Interval; Qw = Q within; Qb = Q between 


3.1.2. Moderating variable analysis degree 

The moderating variable degree consists of four categories: elementary school, junior high school, 
senior high school, and university. The results of the moderating variable analysis as show in Table 6, that the 
effect size means the score of the four degrees is not significantly different from one to another (Qb = 5.048; 
p = 0.168). This finding indicates that the moderating variable degree does not significantly influence the 
effectiveness of technology in language learning compared to traditional language learning. However, the use 
of technology in language learning throughout the four categories of degrees is proven to be effective compared 
to traditional language learning. The use of technology in language learning is most effective in senior high 
school (g = 0.979; p < 0.050), followed by junior high school (g = 0.558; p < 0.050), university 
(g = 0.536; p < 0.010), and elementary school (g = 0.484; p < 0.010). 


3.1.3. Sample size 

The study's moderating variable sample size consists of two categories: the studies with small and big 
sample sizes. The results of the moderating variable analysis as show in Table 6, that the effect size means 
score in both categories of the moderating variable sample size is not significantly different 
(Qb = 1.085; p = 0.297). This finding indicates that the sample size does not influence the effectiveness of 
technology in language learning compared to traditional language learning. Despite that, the effect size means 
score of the studies with small sample sizes (g = 0.605; p < 0.010) is higher than that of the studies with big 
sample sizes (g = 0.553; p < 0.010), the differences in the effect size mean score between the two categories of 
the moderating variable sample size has been confirmed insignificant. 


3.1.4. Country 

The moderating variable country consists of eight categories: Indonesia, Iran, Jordania, Malaysia, 
Pakistan, Turkey, and the United States of America. The results of the moderating variable analysis as show in 
Table 6, uncover that the effect size means the score of the studies in the eight categories has been significantly 
different from one to another (Qb = 52.433; p < 0.010). These findings indicate that the research locations 
(countries) significantly influence the effectiveness of technology in language learning compared to traditional 
learning. From the eight categories of the moderating variable country, the studies that have been conducted in 
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Jordania earn the highest mean score and significant effect size of all countries (g = 1.180; 
p < 0.010), followed by Pakistan (g = 0.696; p < 0.010) and Iran (g = 0.649; p < 0.10). These findings show 
that the use of technology in language learning is effective compared to traditional learning in the three 
countries. On the contrary, the studies that have been conducted in Indonesia, Turkey, and the United States of 
America have been confirmed insignificant. Thus, the statement implies that the use of technology in language 
learning has been confirmed ineffective compared to traditional learning in countries other than Jordania, 
Pakistan, and Iran. 


3.1.5. Skills under measurement 

The study's variable moderating skills under measurement consist of seven categories: general 
proficiency, listening, reading, writing, vocabulary, and sign language. The results of the moderating variable 
analysis as show in Table 4, that the effect size means a score of the seven categories under the variable 
moderating skills under measurement has been significantly different from one to another (Qb = 18.308; 
p < 0.050). These results thus indicate that the skills under measurement influence the effectiveness of 
technology compared to traditional language learning. Then, from the seven categories of the variable 
moderating skills under measurement, technology has been found most effective in sign language (g = 0.725; 
p < 0.010). In addition, the use of technology in mathematical learning has also been found effective for 
exercising vocabulary mastery (g = 0.659; p < 0.010), writing skills (g = 0.498; p < 0.050), and general skills 
(g = 0.391; p < 0.010). On the contrary, to exercise other skills, such as listening, and reading, technology has 
been confirmed ineffective. 


3.1.6. Technology in use 

The variable moderating technology in use consists of five categories: computer, mobile application, 
online media, social media, and other media/technology. The results of the moderating variable analysis as 
show in Table 4, that the effect size means a score of the five categories under the technology in use moderating 
variable has been significantly different from one to another (Qb = 30.197; p < 0.010). The statement indicates 
that the kind of technology in use influences the effectiveness of the use of technology in language learning 
compared to traditional learning. From the five categories under the variable moderating technology in use, the 
category mobile application has been the most effective compared to the other media/technology (g = 0.724; p 
< 0.010), followed by online media (g = 0.523; p < 0.010), computer (g = 0.496; p < 0.010), and other 
media/technology (g = 0.261; p < 0.050). On the contrary, the use of social media in language learning has 
been confirmed ineffective compared to traditional learning. 


3.1.7. Evaluation of publication bias 

The funnel plot from the 43 studies shows that all plots of effect size have inclined to shape 
symmetrical patterns as shown in Figure 2. The statement indicates that there is not any issue of publication 
bias in the data that have been used for the study. Then, the results of Egger’s test are z = 1.515 and p = 0.130, 
which confirm that the funnel plot that has been shaped is symmetrical. According to Rothstein et al. [41], 
when the fail-safe N value is higher than 5 K + 10 (K = a number of individual studies), there is no publication 
bias within the meta-analysis. In the current study, K = 43 and therefore 5(43) + 10 = 225. Afterwards, the fail- 
safe N value that has been earned in the study is 3464 with a target of significance 0.050 and p < 0.001. These 
results also confirm that there has been no publication bias issue in the meta-analysis study. Hence, the 
publication bias in the meta-analysis study is not found. 


3.2. Discussions 

Technology has developed rapidly [42], [43]. Consequently, technology influences all aspects, 
including education [44]-[47]. Hence, the use of computers, smartphones, and laptops in the learning process 
has been a familiar scene in education [48]. In other words, technology has made something challenging to be 
done quickly. At the same time, technology can solve the issue of representation and the issue of time and 
space [49], [50]. The statement was deemed more prevalent when the world was hit by the COVID-19 
pandemic [51], [52]. This situation has urged all countries to integrate technology into their education [53]. As 
a result, the conventional practice has been replaced by the virtual practice [54], [55], the offline classroom has 
been turned into the online classroom, the face-to-face meeting has been turned into the screen-to-screen 
meeting, and the blackboard has been replaced by the monitor [49], [50]. 

Many studies show that technology has delivered positive results to the learning process output [56], 
including one in language learning [15], [19], [57]. However, the implementation of technology in the learning 
process entails numerous requirements, such as facilities, teacher competencies, and student literacy [58]. 
Thereby, it can be immediately concluded that technology has delivered positive results to the learning process 
[15], [18], [28]. Hence, the better the preparedness of all supporting requirements is, the better the impact will 
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be [51], [53]. On the contrary, if the requirements are not fulfilled, then there is a possibility that the technology 
will not deliver a significant impact on the learning process output quality [59]. Thus, something logical will 
be found in the various impact of technology on language learning quality. It makes sense that the findings 
from each country's data show different impacts since each country has specific educational criteria [60]. 
However, in general, it is found that technology is able to support language learning output quality. 

Behind the general description of the role of technology in the learning process, several pieces of 
information have been more detailed. These facts show that the technology can be implemented in all 
educational degrees and display positive results [14], [15], [24], [26]. The nature of technology learning that 
can pack the learning into more variative activities has been alleged as the cause behind these findings [53]. 
For example, native speaker-based and culture-based learning [61], [62] have made learning more attractive 
[63]. Such implementation will be successful if the substance and the content pay attention to the profile of the 
students [48]. Implementing technology-based learning designed based on the characteristics of the students 
will improve the learning process output quality in each educational degree. 

The number of students in a classroom has also been discovered to have an impact. The study's sample 
size supports the assertion. The data indicate that implementing technology in both small and large classrooms 
has a significant positive effect. The optimal number of students in a classroom is a topic of ongoing debate 
[64], and there have yet to be any clear conclusions about which size is better [65]. Another crucial aspect of 
the learning process has an influence over the language learning process [66]. The classroom size in language 
lessons will still be maximized if the teachers can appropriately set the learning process [67]. Thereby, 
implementing technology-based language learning can be recommended for small and big classrooms. 

Optimizing technology for language learning is another equally intriguing topic of discussion. There 
is evidence that certain types of language proficiency have benefited from using technology, but the technology 
has had no discernible effect on the other skills. During the learning process, every competency exhibits 
particular characteristics [68], [69]. Reading is an example of a competency whose characteristics make 
students uneasy when the technological intervention is implemented [70], [71]. Reading on the device is less 
effective [71] as the students feel more comfortable reading through papers [72]-[74]. If students view 
instructional materials on a screen for an extended time, they may develop eyesight problems [75], [76]. 
Implementing technology in language learning can be accomplished by focusing on the competencies to be 
enhanced. The analysis results indicate that technology contributes significantly to enhancing language 
proficiency, writing skills, vocabulary mastery, and sign language comprehension. 
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Figure 2. Funnel plot of the 43 independent studies 


4. CONCLUSION 

In conclusion, the effectiveness of technology in language learning has been demonstrated by the 
authors, with technology surpassing traditional language learning methods. The impact of technology on 
language learning has been categorized as "Moderate" and "Significant," highlighting its substantial 
contribution to the learning process. Several factors that influence the effectiveness of technology in language 
learning have been identified, including the geographical location of the studies (countries), the specific 
language proficiency being measured, and the type of technology employed. Importantly, no significant impact 
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has been found on the effectiveness of technology in language learning due to educational degree and sample 
size. To advance our understanding further, future research should delve into the underlying factors 
contributing to the varying rates of technology's effectiveness in language learning across different countries. 
This investigation can shed light on the nuanced aspects of technology integration in diverse educational 
contexts. Practitioners and educators in the field of language learning are encouraged to carefully select the 
appropriate technology for enhancing specific language proficiency skills. Tailoring technology solutions to 
match the desired language competency can optimize the learning experience and outcomes. As technology 
continues to evolve, its role in language learning is expected to expand and diversify. By continually exploring 
the dynamic relationship between technology and language acquisition, the full potential of technology in 
language learning can be harnessed. 


APPENDIX 
Table 3. Summary of studies included in the meta-analysis (part 1) 
No. Study g SE Type Sample size Grade level 
1. Hosseinpour et al. [21] 0.228 0.274 Journal Small University 
2 Abdulrahman et al. [22] 0.095 0.257 Journal Small Senior high school 
3. López [23] 1.084 0.375 Journal Small University 
4. Mellati and Khademi [29] 1.286 0.262 Journal Big University 
5. Hussain [6] 0.612 0.287 Journal Small Junior high school 
6. Enayati and Gilakjani [5] 0.657 0.261 Journal Small University 
7. Farooq et al. [24] 0.300 0.306 Journal Small University 
8. Schechter et al. (1) [13] 0.261 0.220 Journal Big Elementary school 
9. Schechter et al. (2) [13] 0.042 0.219 Journal Big Elementary school 
10. Schechter et al. (3) [13] 0.343 0.221 Journal Big Elementary school 
11. Parvez etal. (1A) [14] 0.733 0.149 Journal Big Elementary school 
12. Parvez et al. (1B) [14] 0.796 0.150 Journal Big Elementary school 
13. Parvez et al. (1C) [14] 0.526 0.147 Journal Big Elementary school 
14. Parvez et al. (2A) [14] 1.105 0.155 Journal Big Elementary school 
15. Parvez et al. (2B) [14] 0.797 0.150 Journal Big Elementary school 
16. Parvez et al. (2C) [14] 0.414 0.146 Journal Big Elementary school 
17. Elverici (1) [15] 2.899 0.506 Journal Small Senior high school 
18. Elverici (2) [15] -0.254 0.351 Journal Small Senior high school 
19. Kurt (1) [16] 0.675 0.311 Journal Small Elementary school 
20. Kurt (2) [16] 0.511 0.307 Journal Small Elementary school 
21. Awada [25] 2.056 0.342 Journal Small University 
22. Kashani et al. (1) [17] 0.690 0.256 Journal Big University 
23. Kashani et al. (2) [17] 0.386 0.251 Journal Big University 
24. Kashani et al. (3) [17] 0.149 0.249 Journal Big University 
25. Kashani et al. (4) [17] 0.023 0.249 Journal Big University 
26. Kashani et al. (5) [17] -0.049 0.249 Journal Big University 
27. Kashani et al. (6) [17] 0.327 0.250 Journal Big University 
28. — Liu et al. [26] 0.527 0.216 Journal Big Junior high school 
29. Yeşilbağ and Korkmaz (1) [18] -0.092 0.259 Journal Small Elementary school 
30. Yeşilbağ and Korkmaz (2) [18] 0.163 0.259 Journal Small Elementary school 
31. Yeşilbağ and Korkmaz (3) [18] 0.081 0.259 Journal Small Elementary school 
32. Yeşilbağ and Korkmaz (4) [18] 0.627 0.265 Journal Small Elementary school 
33. Yeşilbağ and Korkmaz (5) [18] 0.246 0.260 Journal Small Elementary school 
34. Abu-Hardan et al. (1) [19] 1.431 0.420 Journal Small Senior high school 
35. Abu-Hardan et al. (2) [19] 1.198 0.438 Journal Small Senior high school 
36. Abu-Hardan et al. (3) [19] 0.777 0.506 Journal Small Senior high school 
37. Abu-Hardan et al. (4) [19] 0.665 0.572 Journal Small Senior high school 
38. Abu-Hardan et al. (5) [19] 1.277 0.245 Journal Big Senior high school 
39. BavaHarji et al. (1) [20] 0.514 0.211 Journal Big University 
40. BavaHarji et al. (2) [20] 0.616 0.213 Journal Big University 
41. BavaHarji et al. (3) [20] 0.594 0.212 Journal Big University 
42. Alemi et al. [27] 0.716 0.314 Journal Small University 
43. Alfaleh [28] -0.447 0.480 Journal Small University 
Table 4. Summary of studies included in the meta-analysis (part 2) 
No. Study Country Skills under measurement Technology in use 
1. Hosseinpour et al. [21] Tran Writing Blended learning 
2. Abdulrahman et al. [22] Indonesia Listening Podcasts 
3. López [23] Spanyol Vocabulary Technology-based approach 
4. Mellati and Khademi [29] Iran Writing Computer-mediated communication 
5. Hussain [6] Indonesia _ Vocabulary Wiki 
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Table 4. Summary of studies included in the meta-analysis (part 2) (continue) 


No. Study Country Skills under measurement Technology in use 

6. Enayati and Gilakjani [5] Tran Vocabulary CALL 

7. Farooq et al. [24] Pakistan General Computer-mediated communications 
8. Schechter et al. (1) [13] USA General Blended Learning 

9. Schechter et al. (2) [13] USA Vocabulary Blended Learning 

10. Schechter et al. (3) [13] USA Comprehension Blended Learning 

16. Parvez et al. [2C) [14] Pakistan Sign language Mobile application 

17. Elverici (1) [15] Turki Social presence Social media 

18. Elverici (2) [15] Turki Soc. med. attitude points Social media 

19. Kurt (1) [16] Turki Listening Technology-mediated tasks 
20. Kurt (2) [16] Turki Reading and Writing Technology-mediated tasks 
21. Awada [25] Lebanon Critique writing skills Mobile application (WhatsApp) 
22. Kashani etal. (1) [17] Malaysia Writing-content Blogging 

23. Kashani et al. (2) [17] Malaysia Writing-Organization Blogging 

24. Kashani et al. (3) [17] Malaysia writing-vocabulary Blogging 

25. Kashani et al. (4) [17] Malaysia _writing-language use Blogging 

26. Kashani et al. (5) [17] Malaysia __ writing-mechanics Blogging 

27. Kashani et al. (6) [17] Malaysia Writing Blogging 

28. Liu etal. [26] China Vocabulary CALL 

29. Yeşilbağ and Korkmaz (1) [18] Turki Listening Voki application 

30. Yeşilbağ and Korkmaz (2) [18] Turki Reading Voki application 

31. Yeşilbağ and Korkmaz (3) [18] Turki Writing Voki application 

32. Yeşilbağ and Korkmaz (4) [18] Turki Speaking Voki application 

33. Yeşilbağ and Korkmaz (5) [18] Turki General Voki application 

34. Abu-Hardan et al. (1) [19] Jordan Vocabulary-reasoning TPACK-based instructional 
35. Abu-Hardan et al. (2) [19] Jordan Vocabulary-decoding TPACK-based instructional 
36. Abu-Hardan et al. (3) [19] Jordan Vocabulary-inferring TPACK-based instructional 
37. | Abu-Hardan et al. (4) [19] Jordan Vocabulary TPACK-based instructional 
38. | Abu-Hardan et al. (5) [19] Jordan Reading TPACK-based instructional 
39. BavaHarji et al. (1) [20] Tran Reading Instructional videos 

40. BavaHarji et al. (2) [20] Tran Vocabulary Instructional videos 

41. BavaHarji et al. (3) [20] Tran General Instructional videos 

42. Alemi et al. [27] Tran Vocabulary Mobile phones 

43. Alfaleh [28] USA Reading Internet-based applications 


Table 5. Descriptive statistics of the included studies 


Moderator variable 


Identified categories 


Counts (%) 


Sample size 


Grade level of participants 


Country 


Skills under measurement 


Technology in Use 


Small (N < 30) 
Big (N > 30) 
Elementary school 
Junior high school 
Senior high school 
University 
Indonesia 

Tran 

Jordania 

Malaysia 

Pakistan 

Turkey 

United States of America 
Others 

Umum 

Listening 

Reading 

Writing 
Vocabulary 

Sign language 
Others 

Computer 

Mobile application 
Online media 
Social media 
Others 


22 (51.16%) 
21 (48.84%) 
16 (37.21%) 
2 (4.65%) 
8 (18.60%) 
17 (39.53%) 
2 (4.65%) 
7 (16.28%) 
5 (11.63%) 
6 (13.95%) 
7 (16.28%) 
9 (20.93%) 
4 (9.30%) 
3 (6.98%) 
5 (11.63%) 
3 (6.98%) 
5 (11.63%) 
10 (23.26%) 
11 (25.58%) 
6 (13.95%) 
3 (6.98%) 
9 (20.93%) 
7 (16.28%) 
16 (37.21%) 
3 (6.98%) 
8 (18.60%) 
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